TLDR: This PR enables content-defined chunking (FastCDC) for large uploads/downloads to remote cache, saving ~40% storage, ~50% upload bandwidth, and making builds faster by deduplicating similar artifacts across builds.

RELNOTES[NEW]: Added --experimental_remote_cache_chunking flag to read and write large blobs to/from the remote cache in chunks. Requires server support.

Motivation

Actions like GoLink and CppLink produce very large output files that are often similar between builds. A small source change can cause a cache miss, wasting storage, bandwidth, and time on nearly-identical artifacts.

Content-Defined Chunking (CDC) addresses this by splitting files at content-determined cut points. Because cut points are derived from the file content itself, small changes, even ones that shift bytes around, tend to affect only a few chunks. This makes action outputs effectively incremental: even though the action must re-run, the upload, download, and storage costs shrink dramatically.

Results

Benchmarked across the last 50 commits of the BuildBuddy repo (server and client on the same host):

Key takeaways:

• ~50% less data uploaded (52 GB vs 90 GB)
• ~40% smaller disk cache (147 GB vs 247 GB)
• Download size is mostly unchanged (~0.2% increase) because we don't yet store downloaded chunks in the output base. Using a disk cache is recommended for full benefit; output-base chunk reuse is planned.
• RPC count increases as expected since requests become smaller and more granular.
• faster builds (depends on conditions, like cache async, compression, & network speed)

Additional benefits: better load balancing across distributed clusters (fewer long-running RPCs) and more granular retries on unstable networks.

How It Works

Write path:

1. Check if blob exceeds the chunking threshold.
2. Run FastCDC to compute chunk boundaries.
3. Call FindMissingBlobs to identify which chunks the server already has.
4. Upload only the missing chunks.
5. Call SpliceBlob to register the blob-to-chunks mapping on the server.

Read path:

1. Check if blob exceeds the chunking threshold.
2. Call SplitBlob to get the chunk list for this blob.
3. Download and reassemble the chunks.

If --disk_cache is enabled, previously downloaded chunks are served locally.

Dependencies

Depends on the APIs added in #28614